(Azure) Resolve invalid JSON Object under properties field#1015
(Azure) Resolve invalid JSON Object under properties field#1015mattsp1290 merged 5 commits intoDataDog:masterfrom
Conversation
mattsp1290
left a comment
mattsp1290
left a comment
There was a problem hiding this comment.
Overall this seems like a useful change. Thanks for putting this together. We'd just like to see a little simplification in the added function to reduce the surface of any issues.
| fixPropertiesJsonString(record) { | ||
| try { | ||
| // Check if properties field is a string | ||
| if (typeof record.properties === 'string') { | ||
| // If it is a string, check if it is a malformed JSON String | ||
| // By checking for ':' | ||
| if (record.properties.includes("':'")) { | ||
| // If the check is true, find / replace single quotes | ||
| // with double quotes, to make a proper JSON | ||
| // which is then converted into a JSON Object | ||
| record.properties = JSON.parse(record.properties.replace(/'/g, '"')); | ||
| return record; | ||
| }else { | ||
| return record; | ||
| } | ||
| } else { | ||
| return record; | ||
| } | ||
| } catch { | ||
| this.context.error('Unable to fix properties field to JSON Object'); | ||
| return record; | ||
| } | ||
| } |
There was a problem hiding this comment.
| fixPropertiesJsonString(record) { | |
| try { | |
| // Check if properties field is a string | |
| if (typeof record.properties === 'string') { | |
| // If it is a string, check if it is a malformed JSON String | |
| // By checking for ':' | |
| if (record.properties.includes("':'")) { | |
| // If the check is true, find / replace single quotes | |
| // with double quotes, to make a proper JSON | |
| // which is then converted into a JSON Object | |
| record.properties = JSON.parse(record.properties.replace(/'/g, '"')); | |
| return record; | |
| }else { | |
| return record; | |
| } | |
| } else { | |
| return record; | |
| } | |
| } catch { | |
| this.context.error('Unable to fix properties field to JSON Object'); | |
| return record; | |
| } | |
| } | |
| fixPropertiesJsonString(record) { | |
| // Check if properties field is a malformed JSON String | |
| if (Object.hasOwn(record, 'properties') && (typeof record.properties === 'string') && (record.properties.includes("':'"))) { | |
| try { | |
| // If the check is true, find and replace single quotes | |
| // with double quotes, to make a proper JSON | |
| // which is then converted into a JSON Object | |
| parsedProperties = JSON.parse(record.properties.replace(/'/g, '"')); | |
| record.properties = parsedProperties; | |
| } catch { | |
| this.context.error('Unable to fix properties field to JSON Object'); | |
| } | |
| return record; | |
| } |
This function has a lot of return paths that all seem to do the same thing. This could be simplified a bit to have a single return and combine our checks into a single statement to verify whether or not record.properties has one of these malformed objects.
We'll need to do a little verification on our end to make this work with live data and double check the behavior of nested message and level fields.
There was a problem hiding this comment.
@mattsp1290 I went ahead and took the changes that you've recommended. I added a couple of return clauses that ensures:
- When the if statement fails, we should return the record as is
- If the conversion fails, we should return the record as is
fixPropertiesJsonString(record) {
// Check if properties field is a malformed JSON String
if (Object.hasOwn(record, 'properties') && (typeof record.properties === 'string') && (record.properties.includes("':'"))) {
try {
// If the check is true, find and replace single quotes
// with double quotes, to make a proper JSON
// which is then converted into a JSON Object
record.properties = JSON.parse(record.properties.replace(/'/g, '"'));
return record;
} catch {
this.context.error('Unable to fix properties field to JSON Object');
return record;
}
} else {
return record;
}
}I have also added a couple of additional test cases, that will test nested objects.
…tion Co-authored-by: Matt Spurlin <matt.spurlin@datadoghq.com>
… nested objects and no properties field
d1g1tal-lore-34
force-pushed
the
jcruz/fix-log-properties
branch
from
f17d894 to
9867288
Compare
mattsp1290
left a comment
mattsp1290
left a comment
There was a problem hiding this comment.
I have been able to validate and will make the remaining required changes in a follow up PR.
|
You can track this PR for when this will be available via the deploy to azure button in this repo |
2 participants
What does this PR do?
Added a function call
fixPropertiesJsonString, that works on converting malformed JSON String that is part of the properties field into a proper JSON object. This issue is documented in the follow Microsoft Post.Motivation
This will allow Datadog Log Processor to extract the proper message and log level field. This fixes issue #1014
Testing Guidelines
It has been tested in two manners:
EventhubLogHandler Fix Properties Json StringAdditional Notes
This issue is documented in the follow Microsoft Post.
Types of changes
Check all that apply